Changes

Problem (kind of) solved.

The image snapshots are now only tested for ubuntu-22.04, the exact version I used for generating them. The remaining tests are run for windows-latest, macOS-latest and ubuntu-22.04.

Comments

It could be a good idea to do one of these two in the near future:

1. Transfer the snapshot tests to the same system @mdingemanse is using (macOS?). If we get together in the same room, that shouldn't take long.
   or
2. Abandon using snapshots. We will still be more reproducible than 90% of academic code 😅

On the larger question: Since the point of the snapshot is not pixel-perfect alignment across platforms anyway —OS difference in viewports, available fonts & other settings mean this is out of reach— I don't think we should be comparing them across OSs.

I also still don't get why this results in an error as opposed to a snapshot that we have to approve manually as "similar enough" in testthat output.

If that error cannot be avoided, snapshots are useless to us.

On the larger question: Since the point of the snapshot is not pixel-perfect alignment across platforms anyway —OS difference in viewports, available fonts & other settings mean this is out of reach— I don't think we should be comparing them across OSs.

I fully agree.

I also still don't get why this results in an error as opposed to a snapshot that we have to approve manually as "similar enough" in testthat output.

Computers are notoriously bad at common sense. Testing "similar enough" as opposed to "identical" would mean opening a whole new can of worms¹.

If that error cannot be avoided, snapshots are useless to us.

Not sure if useless is the word I'll use, but indeed, they become quite annoying.